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Abstract. We present two new hybrid techniques that replace the synchronized product used in the 
automata-theoretic approach for LTL model checking. The proposed products are explicit graphs 
of aggregates (symbolic sets of states) that can be interpreted as Biichi automata. These hybrid 
approaches allow on the one hand to use classical emptiness-check algorithms and build the graph 
on-the-fly, and on the other hand, to have a compact encoding of the state space thanks to the 
symbolic representation of the aggregates. The Symbolic Observation Product assumes a globally 
stuttering property (e.g., LTL\ X) to aggregate states. The Self-Loop Aggregation Product does not 
require the property to be globally stuttering (i.e., it can tackle full LTL), but dynamically detects 
and exploits a form of stuttering where possible. Our experiments show that these two variants, 
while incomparable with each other, can outperform other existing approaches. 

1 Introduction 

Model checking for Linear-time Temporal Logic (LTL) is usually based on converting the property into 
a Biichi automaton, composing the automaton and the model (given as a Kripke structure), and finally 
checking the language emptiness of the composed system [24] . This verification process suffers from a well 
known state explosion problem. Among the various techniques that have been suggested as improvement, 
we can distinguish two large families: explicit and symbolic approaches. 

Explicit model checking approaches explore an explicit representation of the product graph. 
A common optimization builds the graph on-the-fly as required by the emptiness check algorithm: the 
construction stops as soon as a counterexample is found [5]. 

Another source of optimization is to take advantage of stuttering equivalence between paths in the 
Kripke structure when verifying a stuttering-invariant property [9] : this has been done either by ignoring 
some paths in the Kripke structure [14], or by representing the property using a testing automaton [13]. 
To our knowledge, all these solutions require dedicated algorithms to check the emptiness of the product 
graph. 

Symbolic model checking tackles the state-explosion problem by representing the product au- 
tomaton symbolically, usually by means of decision diagrams (a concise way to represent large sets or 
relations). Various symbolic algorithms exist to verify LTL using fixpoint computations (see [10,22] for 
comparisons and [15] for the clarity of the presentation). As-is, these approaches do not mix well with 
stuttering invariant reductions or on-the-fly emptiness checks. 

However explicit and symbolic approaches are not exclusive, some combinations have already been 
studied [2, 11, 21, 16] to get the best of both worlds. They are referred to as hybrid approaches. 

Most of these approaches consist in replacing the KS by an explicit graph where each node contains 
sets of states of the KS (called aggregates throughout this paper), that is an abstraction of the KS 
preserving properties of the original KS. In [2] for instance, each aggregate contains states that share 
their atomic proposition values, and the successor aggregates contain direct successors of the previous 
aggregate, thus preserving LTL but not branching temporal properties. In [11] this idea is taken one 
step further in the context of stuttering invariant properties, and each aggregate now contains sets of 
consecutive states that share their atomic proposition values. In both of these approaches, an explicit 
product with the formula automaton is built and checked for emptiness, allowing to stop early (on-the-fly) 
if a witness trace is found. 

The approach of [21] is a bit different, as it builds one aggregate for each state of the Biichi automata 
(usually few in number) , and uses a partitioned symbolic transition relation to check for emptiness of the 
product, thus resorting to a symbolic emptiness-check (based on a symbolic SCC hull computation). 

The hybrid approaches we deflne in this paper are based on explicit graphs of aggregates (symbolic 
sets of states) that can be interpreted as Biichi automata. With this combination, we can use classi- 
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cal cmptincss-chcck algorithms and build the graph on-thc-fly, moreover the symbolic representation of 
aggregates gives us a compact encoding of the state space along with efficient fixpoint algorithms. 

The first technique we present extends the Symbolic Observation Graph (SOG) technique [11,16] 
(which itself can be seen as a specialization of the work of Biere et al. [2] for stuttcring-invariant prop- 
erties). Given a property, only a subset of atomic propositions of the system need to be observed. The 
SOG approach aggregates consecutive states of the Kripke structure that share the same values for the 
observed atomic propositions. The SOG is an aggregated Kripke structure that is stuttering equivalent to 
the original Kripke structure. We combine this principle with an idea presented by Kokkarinen et al. [17] 
in the context of partial order reductions: as we progress in the Biichi automaton, the number of atomic 
propositions to observe diminishes and allows hirther aggregation. Wc call this new graph a Symbolic 
Observation Product (SOP), because it replaces the product between the Kripke structure and the Biichi 
automaton in the explicit approach. 

The second technique we present also defines an aggregation graph which is a product: the Self- Loop 
Aggregation Product (SLAP). It uses a different aggregation criterion based on the study of the self-loops 
around the current state of the Biichi automaton. Roughly speaking, consecutive states of the system are 
aggregated when they arc compatible with the labels of self-loops. Unlike the previous approach, SLAP 
is not limited to stuttering-invariant properties. It dynamically allows to stutter according to a boolean 
formula computed as the disjunction of the labels of self-loops of the automata. 

This paper is organized as follows. Section 2 introduces our notations, presents the basic automata- 
theoretic approach and compares it to the (existing) SOG approach. Sections 3 and 4 define our two 
new hybrid approaches: SOP and SLAP. We explain how we implemented these approaches and evaluate 
them in Section 5. 

2 Preliminaries 

2.1 Boolecin Formulcis 

Let AP be a set of (atomic) propositions, and let B = {±, T} represent Boolean values. We denote B(AP) 
the set of all Boolean formulas over AP, i.e., formulas built inductively from the propositions AP, B, 
and the connectives A, V, and If AP' C AP, then we have B(AP') C B(AP) by construction. For 
any formula /, we will note FV{f) (for Free Variables) the set of propositions that occurs in /, e.g., 
FV{bV^a) = {a,b}. 

An assignment is a function p : AP B that assigns a truth value to each proposition. We denote 
B^^ the set of all assignments of AP. Given a formula / G B(AP) and an assignment p £ B'*^^, we denote 
p{f) the evaluation of / under p.^ In particular, we will write p ^ / iff p is a satisfying assignment for 
/, i.e., p\= f ^ p{j) = T. The set B*(AP) = {/ e B(AP) | 3p e B^'^^p h /} contains all satisfiable 
formulas. 

We will use assignments to label the states of the model we want to verify, and the propositional 
functions will be used as labels in the automaton representing the property to check. The intuition is that 
a behavior of the model (a sequence of assignments) will match the property if we can find a sequence of 
formulas in the automaton that are satisfied by the sequence of assignments. 

We will write p = p' iff p\e = p'\e^ where p\E denotes the restriction of the function p to the domain 
E. This means that assignments p and p' match on the propositions E. 

It is sometimes convenient to interpret an assignment p as a formula that is only true for this as- 
signment. For instance the assignment {a i->- T,6 h-t- T,c i->- _L} can be interpreted as the formula 
a A 6 A -ic. So we may use an assignment where a formula is expected, as if we were abusively assuming 
that B^P c B(AP). 

2.2 TGBA 

A Transition-based Generalized Biichi Automaton (TGBA) is a Biichi automaton in which generalized 
acceptance conditions are expressed in term of transitions that must be visited infinitely often. The reason 

we use these automata is that they allow a more compact representation of properties than traditional 
Biichi automata (even generalized Biichi automata) [7] without making the emptiness check harder [6]. 

Definition 1 (TGBA). A Transition-based Generalized Biichi Automata is a tuple A = (AP, Q, J^, 5, q°} 
where 



* This can be defined straightforwardly as p{f Ag) = p{f) A p{g), p{~'f) = ~'P{f), etc. 
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— AP is a finite set of atomic propositions, 

— Q is a finite set of states, 

— T is a finite and non-empty set of acceptance conditions, 

— (5 C Q X B*(AP) X 2-^ X Q is a transition relation. We will commonly denote qi •^'"°> 92 an element 

(gi,/, 00,92) G S, 

— € Q is the initial state. 

An execution (or a run) of A is an infinite sequence of transitions tt = (si, /i, aci, c?i) • • • 

{si, fi,aci,di) ■ • ■ E S'^ with si = g'' and Vi,di ~ Si+i- We shall simply denote it as tt = si ^^'°'^'"> 

52 ^^'°''^^) S3 • • • . Such an execution is accepting iff it visits each acceptance condition infinitely often, i.e., 
if Va e Vi > 0, 3j > i, a e acj. We denote Acc(j4) C J" the set of accepting executions of A. 

A behavior of the model is an infinite sequence of assignments: p\p2P?> ■ • • G (B^^)", while an execution 
of the automaton A is an infinite sequence of transitions labeled by Boolean formulas. The language of A, 
denoted /^(A), is the set of behaviors compatible with an accepting execution of A: C,{^A) = {p\p2 ■ • ■ S 

(B^P)'^ I 3si S2 • • • e Acc(A) and Vi > 1, /o« h .f^} 

The non-emptiness constraint on T was introduced into definition 1 to avoid considering J" = as 
a separate case. If no acceptance conditions exist, one can be artificially added to some edges, ensuring 
that every cycle of the TGBA bears one on at least an edge. Simply adding this artificial acceptance 
condition to all edges might seriously hurt subsequent verification performance, as some emptiness-check 
algorithms are sensitive to the position of acceptance conditions. 

Fig. la represents a TGBA for the LTL formula a\ih. The black dot on the self-loop q\ "'"'^*'^> q\ 

denotes an acceptance conditions from = {•}. The labels on edges {ab,b,T) represent the Boolean 
expressions over AP = {a, b}. There are many other TGBA in Fig. 1, that represent product constructions 
of this TGBA and the Kripke Structure of Fig. lb. 
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A language C{A) is stuttering-invariant if any letter of a word can be repeated without affecting 
its membership to the language. In other words, £{A) is stuttering-invariant iff for any finite sequence 
u e (B^^)*, any assignment p G B^^, and any infinite sequence v € (B^^)" we have upv G JC{A) <^=^ 
uppv <E jC{A). 

Two sequences wi and W2 are stuttering equivalent iff they are equal after removing all repeated letters. 
Two languages jC{A) and JC{B) are stuttering equivalent iff any word of JC{A) is stuttering equivalent to 
a word of jC{B) and vice versa. 

LTL\X is the set of LTL formulas that do not use the X (next-time) operator. It is known that 
formulas in LTL\ X describe stuttering-invariant properties (i.e., the language of the corresponding TGBA 
is stuttering-invariant), and that any stuttering-invariant property can be expressed in LTL\ X [19]. 
The TGBA of Fig. 1 corresponds to the LTL formula a U 6 and consequently has a stuttering-invariant 
language. 

2.3 Kripke Structure 

For the sake of generality, we use Kripke Structures (KS for short) as a framework, since the formalism 
is well adapted to state-based semantics. 

Definition 2 (Kripke structure). A Kripke structure is a 4-tuple T = {AP , F, X, A, sq) where: 

— AP is a finite set of atomic propositions, 

— r is a finite set of states, 

— X : r ^ B^^ is a state labeling function, 

— A C r X r is a transition relation. We will commonly denote Si S2 the element (si, S2) G A. 

— Sq G r is the initial state. 

Fig. lb represents a Kripke structure over AP = {a,b,c}. The state graph of a system is typically 
represented by a KS whose labeling function gives the truth values of the atomic propositions for a given 
state of the system. The SOG construction of Fig. Ic also represents a KS; it is an aggregated abstraction 
built from Fig. lb by observing only labels a and b. 

We now define a synchronized product for a TGBA and a KS, such that the language of the resulting 
TGBA is the intersection of the languages of the two automata. 

Definition 3 (Synchronized product of a TGBA and a Kripke structure). Let A = (AP', Q, T , 5, (f') 
he a TGBA and T = (AP, F, A, A, Sq) be a Kripke structure over over AP D AP'. 

The synchronized product of A andT is the TGBA denoted by A^T = {PiI',Q^,T,6^,q^) defined 

as: 

— Q^ = Qxr, 

— S^CQ<^x B*(AP) X 2-^ X where 

S<8 = \ (91, si) (92,52) 



Fig. Id represents such a product of the TGBA a U 6 of Fig.la and the Kripke structure of Fig. lb. 

State (so, 90) is the initial state of the product. Since A(so) = o,bc we have A(so) \= ah, successors {51,54} 

of So in the KS will be synchronized through the edge qo -^M^ Qf TGBA with qo- In state (go, S4) 

the product can progress through the qo qi edge of the TGBA, since A(s4) = abc \= b. Successor S5 
of S4 in the KS is thus synchronized with q\. The TGBA state q\ now only requires states to verify T 
to validate the acceptance condition •, so any cycle in the KS from S5 will be accepted by the product. 
The resulting edge of the product bears the acceptance conditions contributed by the TGBA edge, and 
the atomic proposition Boolean formula label that comes from the KS. The size of the product in both 
nodes and edges is bounded by the product of the sizes of the TGBA and the KS. 

The emptiness-check on a TGBA verifies if there exist a cycle that pass through an accepting edge 
(with the black dot). All of the TGBA product constructions in Fig. 1 agree in having a non-empty 
language, since language emptiness is the property these abstractions (SOP, SLAP) guarantee to preserve. 
These specialized synchronized products that are the main contribution of this paper will be discussed 
as they are defined. 



si ^ S2 G A, A(si) = / and | 
3g e B*(AP) s.t. 91 ^ 92 G <5 and A(si) ^ g] 
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2.4 Symbolic Observation Graph (SOG) 

A symbolic observation graph over AP' is an abstraction of a KS over AP (AP' C AP) built to allow 
preservation of stuttering- invariant properties [11, 16]. 

It uses a symbolic data structure to represent sets of states of the KS that have been aggregated. 
The SOG is not a quotient graph, since the predicate wc use to aggregate states is not an equivalence 
relation. Hence its worst case size in number of states (aggregates) is bounded by 2^, while the number 
of successors of each aggregate is bounded by 2^^ — 1. 

However, in practice, particularly when the set of observed propositions AP is small, which the case 
of a typical LTL formula, the SOG is much smaller than the underlying KS. Since the states in each 
aggregate are stored symbolically, the size of these aggregates is not necessarily the dominating factor in 
the overall complexity. 

Notations For a set of states a C r and a Boolean formula / 6 B(AP), let us denote by SuccF(a, /) = 
{s' G r \ 3s G a, s ^ s' G A A A(s') \= /}, i.e., the set of the Successors states of a Filtered to keep only 

those satisfying /. 

Furthermore, we denote by ReachF(a, /) the least subset of F satisfying: 

- a C ReachF(a, /) 

- SuccF(ReachF(a,/),/) C ReachF(a,/) 

Definition 4 (Homogeneous aggregate). Let a € 2^ \ {0} be a set of states. We say that a. is a 
homogeneous aggregate with respect to a given subset of atomic propositions AP' C AP iff Vs, s' G 

AP' / 

a, A(s) = A(s'). Furthermore, for a homogeneous aggregate a w.r.t. AP C AP, we write AAP'(a) = 
A(s)|AP' for some state s G a. 

A homogeneous aggregate a w.r.t. AP' is then a set of states that share the same values for atomic 
propositions in AP'. The associated label is the label of one of its states. Obviously, a homogeneous 
aggregate a w.r.t. AP' is homogeneous w.r.t. any AP" C AP'. 

Definition 5 (Symbolic Observation Graph). Let T = (AP, F, A, A, sq) be a KS. A symbolic obser- 
vation graph over AP' C AP of T is the KS over AP' defined as Gap' = (AP', S", A', Z\', ao) satisfying 



a is homogeneous w.r.t. AP' 1 
a = ReachF(a, Aap' (a)) J 
Elements of F' are called aggregates and elements o/B^^' are divergent states. 



1. S' = F'U B^P' with r' = i a G 2^ \ {0} 



2. Va G 5', A' (a) 



Aap' (a) */ a € F' 
l^a z/aGB^P' 

3. A' ={a^a' gF' xF' \a' = ReachF(SuccF(a, A' (a')) \ a, A' (a'))} 

U {o — >■ Z G -T' X B^^ I a contains a cycle and I = A' (a)} 
U {/-)•/ I ZgB^p'} 

4. ao = ReachF({so}, A(so)|AP')- 

Following point 1 of the above Definition, the nodes of a SOG arc of two kinds: (1) homogenous 
aggregates a satisfying a = RcachF(a, AAp'(a)), i.e., if a state s E a then each successor s' of s belongs 

to a as soon as A(s') ^= A(s), and, (2), divergent states, labeled with atomic propositions of AP'. The 
transition relation can be informally explained as follows: three kind of edges can connect the nodes of a 
SOG. If a and a' are two aggregates of F' then a a' iff A'(a) ^ A' (a') and each state s' G F satisfying 
A(s')|AP' = A'(a') and s — >■ s' is in a'. Given a G F' and I a divergent state then a ^ Hff a contains a 
cycle and is labeled with I. Finally each divergent state has a self-loop. 

Fig. Ic represents the SOG built over the KS of Fig. lb by disregarding the value of c. We can 
see in this product one divergent states labeled ah that represents the presence of a cycle in the states 
of its predecessor aggregate {sq, si, S2, S3}. States are aggregated as long as they agree on the value of 
the subset of observed atomic propositions. The SOG is still a KS, that allows to check any stuttering- 
invariant property over the alphabet {a,b}. For instance, its product with the TGBA of a U 6 produces 
the TGBA of Fig. le. Both this abstraction and its product are smaller than their equivalents based on 
the plain Kripke structure of Fig. lb. 

Theorem 1 ([16]). Given a Kripke Structure T defined on AP, then the SOG Gap' of T built over 
AP' C AP preserves any stuttering-invariant property A on AP'. In other words: L{A (8) T) 7^ ^^=> 
C{A(^Gap')t^^- 
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3 Symbolic Observation Product (SOP) 

A SOP is a dynamic extension of the SOG [11,16]. Both approaches focus exclusively on stuttering- 
invariant properties. The SOP is a hybrid synchronized prodiict between the TGBA of a stuttering- 
invariant property and a KS. In this product, the size of the observed alphabet AP' decreases as the 
construction progresses (an idea also presented by Kokkarinen et al. [17] in the context of partial order 
reductions), therefore allowing more aggregations. 



3.1 Definition 



Given a TGBA A = (AP, Q,J^,5,q°), let us define the alphabet FV{q) of a state g G Q as the union of 
the atomic propositions which can be observed from q: i.e., FV{q) = [J FV(/) where 5*{q) 

91 >q-2&6*{q) 

designates the set of transitions reachable from a state q. For instance, FV(^'') = AP. It is clear that for 

any gi Iff^ q2 g S, we have FV(g'i) 3 FV{q2). The set of observed atomic propositions in a given state 
and its future reduces or at worse stays stable as we advance through the automaton. 

Definition 6 (SOP of a TGBA and a KS). Given a TGBA A = {AP' , Q,T,5,q°) and a Kripke 
structure T = {AP , F, X, A, sq) over AP D AP', the Symbohc Observation Product of A and T is the 
TGBA denoted A §iT = (AP', Q_ , J", 5-, qS.) where: 

— Qg = Q'UV where states of the automaton are synchronized with aggregates in Q' and with divergent 

states in T>' : 

a is homogeneous w.r.t. FV(g) 1 
a = ReachF(a, AFv(g) (<i)) J 
V = {(qj) \ q&Qandle B^^'-'^^ 

(91, ai) e Q',{q2,a2) G Q', / = AFv(gi)(oi), 



Q'= (a,a)eQx(2^\{0}) 



U < 



u 



(gi,ai) (92,02) 



{qi,a) -^i^ (52,^2) 



(91,^1) (92,^2) 



3/ e B(AP) s.t. qi ^ 92 e 6, and I \= f 
31' e bP^(«^) s.t. a2 = ReachF(SuccF(ai,Z') \oi,Z') ^ 

(gi,a) e Q', (92,^2) e 'D'Ji = AFv(gi)(a), 

a contains a cycle, I2 = ^i|FV(g2)' 

3/ e B(AP) s.t. qi ^ 92 e (5, and h\= f 
(qiji) G V, (92, /2) G V',l2 = ^i|FV(,2)' 
3/ e B(AP) s.t. 91 ^ 92 G 6, and h\= f 



^ % = (9o,ReachF({so},A(.so)|Fv(g„))) 

Let us explain the intuition behind the three terms of the transition relation. The first rule strongly 
resembles the SOG aggregation rule, except that the aggregate built from the successors of states in ai 
that bear the appropriate label only observes the atomic propositions in FV(92). This rule is the main 
ingredient that allows to observe less atomic propositions as the product progresses, and hence be more 
efficient. The next two rules define the cycle detection routine similar to the SOG, but taking into account 
the reduction of the set of atomic propositions to be observed. 

Fig. If represents a SOP built from our example KS and the TGBA of a U &. Because in state 91 
of the formula the observed alphabet is empty, the SOP aggregates the states of the cycle {54, S5, se, S7} 
visible on the right of the product that uses the SOG (Fig. le). This cycle is then identified by the cycle 
detection rules, and visible as a divergent state in the SOP. 



3.2 Proof of correctness 

Our ultimate goal is to establish that, given a KS and a TGBA, the emptiness of the language of the 
corresponding SOP is equivalent to the emptiness of the language of the original synchronized product 
(see Theorem 2) . This result is progressively demonstrated in the following. We proceed by construction 
i.e., if there exists an accepting run of the SOP then we build an accepting run of the original product 
and vice versa. In order to ease the proof of the first direction, we define a new synchronized product 
having a language stuttering equivalent to that of the original synchronized product (Lemma 1). Then, 
the desired accepting run is built in this new product (Lemma 2 and Lemma 3) and not in the original 
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Fig. 2: A prefix (gi, ai) (92,02) — ^ (^2, ^2) of a run of some SOP ^(g) T (with dijffereni ^ and Tfrom 
Fig. 1) is shown using big elhpses and bended arrows. The straight hnes also shows the underlying connec- 
tions between the states {(71, 52, <Z3, • . •} of the automaton A and between the states {si, S2, ■ ■ ■ ,xi,X2 ■ ■ ■} 
of the Kripke structure T that have been aggregated as ai, 02, ^3 . . . The acceptance conditions have been 
depicted as acj and the labels of the transitions have been omitted for clarity. The dotted ellipses show 
the set of input states (7n(ai), In{a2), In{as)) as used in the proof of Lemma 2. 



synchronized product. Hence, the desired result follows immediately (Corollary 1). Conversely, the proof 
of the second direction (each accepting run of the original product corresponds to an accepting run of 
the SOP) is not based on such a product but is nevertheless facilitated by two intermediate lemmas 
(Lemma 4 and Lemma 5). 

Definition 7 (Stuttering synchronized product of a TGBA and a Kripke structure). Let 

A = {AP, Q,T,S,q^) be a stuttering-invariant TGBA and T = (AP, Z^, A, Zi, sq) he a Kripke structure 
over the same atomic proposition set AP. 

The stuttering synchronized product of A and T is the TGBA denoted by A^T = (AP, Q^,J^, S^,q^) 
defined as: 

- Q^ = Qxr, 

- S~CQ^ X B*(AP) X 2-^ X where 



% =\ (9i>si) (92,52) 



U < (gi,si) ^ (gi,S2) 



si S2 G A, A(si) — f and 
3g e ]B*(AP) s.t. qi 92 e 5 and A(si) \= g 
•si — !> S2 e A, A(si) = / and^ 

A(si) = A(s2) J 



-4 = (5°'^o)- 

Lemma 1. Let A and T he defined as in Definition 7. We have Acc{A®T) 7^ ^^=^ Acc(^ ^ 0. 

Proof. By definition, the product A®T contains all the transitions oi A®T^ and adds only stuttering 

transitions of the form {qi,Si) '^"°'^'> ((7^+1,5^+1) such that qi = qt+i, fi fi+i, and aCi = 0. Hence, 
the language £{A<^T) is stuttering equivalent to the language £{A'Si T). Therefore £{A^T) ^ 
jC{A (8> 7^ 7^ and the lemma follows. □ 



Lemma 2. Let A and T be defined as in Definition 7. Let (qi.ai) (52: 0-2) ^ 5^ be a transition of 
a SOP A®T such that (92,02) G Q' ■ For any state S2 G 02 there exists at least one (possibly indirect) 
ancestor si G ai such that (91, si) (92, ii) (32, t2) — >■ ••■(92, in) — >■ (92,52) is a sequence of the 
stuttering product A^T with Vi, e 02- 

For example, consider transition (91, ai) (92, 02) on Fig. 2, and some state in a2, say S2- Then si G ai 
is an indirect ancestor of S2 s.t. (gi, Si) (92, X2) — ^ (92, S2)- 

Proof. Let us define the set of input states of the aggregate 02 as In{a2) = {s' G 02 | 3s G Oi, s — >■ s' G A}. 
This set cannot be empty since (gi,ai) (92,02)- 

Consider a state S2 G 02. By construction of 02, S2 is reachable from some state in ti G Ln{a2), 
so there exists a path ti ^ i2 S2 in the Kripke structure. Furthermore, all these states 

ti,t2, ■ ■ ■ ,S2 are homogeneous w.r.t. FV(92), and the property is stuttering invariant, so there exists a 
path (92, ii) ^ (92, ^2) ^ • • • — >■ (92, S2) in the stuttering product A^T. 
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Moreover, since ti £ In(a2), there exists a state si in ai such that {qi, si) {q2,ti). 

Consequently, the path (gi,si) (52,^1) — > ('^2,^2) — > • • • — > ((72,52) satisfies the lemma. □ 

Lemma 3. Let A and T be defined as in Definition 7. If there exists a S k.cc{A®T) an infinite run 
accepted by the SOP, then there exists an accepting run n £ Acc{A^T) in the stuttering synchronized 
product. 

Proof. There are two cases to consider: 

1. Either a contains some divergent states, and then, by definition of (5g, a must necessarily have a 
finite prefix made of states from Q' foUowcid by an infinite suffix made of states of T>' and the last 
state of that prefix has an aggregate that contains a cycle. 

Let us denote a = {qi,ai) (92,02) ■■■{qk,ak) {qk+i,h) °'''°"^'> {qk+2,h)--- such an 

accepting run of A(E)T- 

Let Sfe, Sk+i, . ■ ■ , Sk+n-i be a cycle of a^. Applying Lemma 2 from to ai, we can build a (possibly 
larger) sequence TVp = {qi, si) • • ■ {qk, Sfe) of transitions of A^T. Since si e ai, i.e., it belongs to 
the initial aggregate, it is accessible from sq by definition of q^. Therefore Wp can be prefixed by a 
sequence starting from {q^, sq); let tt, be this complete prefix, going from {q°, sq) to Sk). 
Let us now complete ttj to an infinite sequence. Because all the states Sk, Sfc+i , • ■ • , Sfc+n-i are homoge- 
neous w.r.t. FV((7fe), they are also homogeneous w.r.t. FV(gfc+i) for any i > 0. By definition of the stut- 
tering synchronized product, tTs = {qk,Sk) • • • {qk+n, Sk+n) °'"°^"> • • • {qk+i,Sk+{i mod n)) °°'°"^'> 
• • • is a path of AiS^T- 

Consequently, the infinite sequence TTiTTg starts from the initial state, and visits the same acceptance 
conditions of a, thus TTjTTs S Acc{A^T). 

2. Or CT traverses only states from Q'. 

Let us denote tr = (gi,ai) — (92,12) (93,13) • • • such an accepting run of A®>T. Let us 
build an infinite tree in which all nodes (except the root) are states of A<^T. Let us call T the root, at 
depth 0. The set of nodes at depth n > is exactly the finite set of pairs {{qn, s) | s G a^} C Qx F. 
The parent of any node at level 1 is T. For any i > 0, the parent of a node (^j+i, s') with s' G ai+i is 
the node {qi, s) for any state s € such that (g,, s) is a (possibly indirect) ancestor of (gj+i, s') such 
that we observe aci on the path between these two states. We know such a state {qi , s) exists because 
of Lemma 2. As a consequence of this parenting relation, every edge in this tree, except those leaving 
the root, correspond to a path between two states of A<SiT. 

Because the set of nodes at depth n > is finite, this infinite tree has finite branching. By K5nig's 
Lemma it therefore contains an infinite branch. By following this branch and ignoring the first edge, 
we can construct a path of A^T that starts in (91, Si) for some Si € ai, and that visits at least all 
the acceptance conditions acj of a in the same order (and maybe more). To prove that this accepting 
path we have constructed actually occurs in a run of A^T, it remains to show that (91, Si) is a state 
that is accessible from the initial state of A<SiT. 

Obviously 91 = q^ because (91,01) = q% is the initial state of A<SiT. Furthermore we have si € ai, 
so by definition of 9^, (9*^, si) must be reachable from (or equal to) {q^, sq) in A^T. 

□ 

Corollary 1. If there exists a G Acc{A<SiT) an infinite run accepted by the SOP, then there exists an 
accepting run n G Acc(,A T) in the synchronized product. 

Proof. Follows from Lemma 3 and Lemma 1. □ 

Lemma 4. Let A and T be defined as in Definition 7. For a given n and a finite path 7r„ = (go, so) '^°'"^''> 
(91, si) • • • '^""''"''""^ {qn, s„) of A®T, there exists a finite path am = (9^(0), ao) '"''^'°'> (9vp(i), oi) ■ • ■ 
— (9(^(TO),oim) of A®T, with m <n, where 



ip{0) = max 
for i> ip{i) = max I j 



Vk e {0, . . .,j}, qk = qj A A(sfe) ^^=''' X{sj) | 

Vk e {(p{i - 1) + 1, . . . ,j}, qk = qj A A(sfe) ^^='^ X{sj) | 



and {so, s^(o)} ^ ao and for i > 0, {s^(i-i)+i, . • . , Si^(j)} C m 



9 



Proof. Let prove this by induction on the length of the finite path. The property is true for a path of 
length n = by definition of q'^ . Now assume that the lemma is true for some length n and let us consider 

a path Pn+i = (qo, so) ^"'°^°> {qi, si) ■ ■ • "'°'^"> (^n+i, s„+i) of length n+1. By the induction hypothesis, 

we know that there exists a finite path cr^ = ((7y,'(o), ao) > • • • > {Q<fi' (m) j '^m) of 

A<SiT, and a function ip' that correspond to the prefix of length n of 7r„+i . 
We consider the following two cases: 

1. If Qn+i = Qn and A(s„+i) ^Xi?"^ A(s„) then s„+i G by definition of S^. Therefore the path 

o-m =■ (g.^(o),ao) — • ■ • (5vp(m),am}) and the function y defined as Vi < 

m, (^(i) = <^'(i) and <^(m) = n + 1, satisfy the lemma. 

FV(g„) 

2. Otherwise, qn+\ 7^ (Zn or A(s„+i) ^ A(s„). In that case, according the definition of there 

exists an aggregate ajn+\ such that {qn-,am) (an+i,am+i) and s„+i e 0^+1. 

Since n = (p'{m). We can define as Vi < m, = (p'{i) and (/j(m + 1) = n + 1, and build am+i 

by extending a^: am+i = {q<p{o),ao) -^^^ (g,^(i),ai) • • • (g<^(m),am) (^^^(m+i), 

This path satisfies the lemma. 

□ 

Lemma 5. Let A and T be defi/aeA as in Definition 7. If there exists an infinite path tt € Acc(^ (8) T) 
accepting in A®T. Then there exists an accepting path in A®T as well. 

Proof. A®T has a finite number of states, so if Acc(^ ®T)^% then it contains at least one infinite 
path TT G Acc(.4, ® T) that can be represented as a finite prefix followed by a finite cycle that is repeated 
infinitely often. 

Let us denote this lasso-shaped path by tt = (g'o, so) ■ ■ ■ {Qk, Sk) — ^ • ■ • (571, s„) with (g„, s„) = 

Note that because qk, %+i, • • • , 9n is a cycle in A, we have FV(gfe) = FV{qk+i) = . . . = FV(g'„_i). 
We consider two possible cases: 

1. If X{sk) X{sk+i) ^'^=''^ . . . ^Xi?*'^ A(s„_i), these states are homogeneous and they form a cycle. 

We can apply Lemma 4 on prefix {qo,so) • • • {qk,Sk) to build a path Um = iq(p{o),ao) — 

{q^{i),o,i) ■ ■ ■ — "^'"^ {qtp{m),am) such that qk = g<^(m) and Sk G a^. Since the states s^, . . . s„_i 
are homogeneous, we also have {sfe, . . . s„_i} C Um- 

Because am contains a cycle, there exist transitions {qk, am) — ^ {qk+i,l) ""''^^ {qk+2, 1) - ■ ■ "^""^ 
{qn = qk,l) (9fe+i, according to Jg. 

The infinite sequence am — ^ ({qk+i,l) °^''^^> {qk+2, 1) ■ ■ ■ "^""^ {qk, I) -^^^ is accepting and sat- 
isfies the lemma. 

2. Otherwise if the states Sk, ■ ■ ■ , Sn-i are not homogeneous w.r.t. FY{qk), then we can apply Lemma 4 on 
the entire path tt in order to build a path am = {qip{o),ao) — (9<^(i), oi) • • • — '^^'"^S {q<p{i),0'i) ■ ■ ■ 
ac^(m-i)^ (g<p(m), ttm) such that Sk G fl; and Sj, = s„ G am- 

If a; = am then am is lasso-shaped and preserves the acceptance conditions visited by tt. Hence the 
lemma is verified. 

Unfortunately it is possible that the aggregate am and ak are diflferent because they were built from 

different predecessors. In that case, consider the lasso-shaped path, where the cycle has been unrolled 
2^ times. Then applying Lemma 4 allows to build a path with am that, among other states, traverses 
2^-1-1 states of the form {{qk, ai)};eo...2^ with all a; containing the state Sk = Sn- Since an aggregate 
is a subset of F, at least two of these {qk, ai) are equal, and therefore we can construct a lasso-shaped 
accepting run that satisfies the lemma. 

□ 

Theorem 2. Let A he a TGBA, and T he a Kripke structure. The SOP of A and T accepts a run 
if and only if the synchronized product of these two structures accepts a run. In other word, we have 
Acc(^ O 70 7^ <^ Acc(^0T)7^0. 

Proof. <= follows from Corollary 1; follows from Lemma 5. □ 
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4 Self-Loop Aggregation Product (SLAP) 

This section presents a hybrid algorithm that is not restricted to stuttering-invariant properties. It is a 

specialized synchronized product that aggregates states of the KS as long as the TGBA state does not 
change, and no new acceptance conditions are visited. 



4.1 Definition 

The notion of self-loop aggregation is captured by SF{q, ac), the Self-loop Formulas (labeling edges g — > g) 
that are weaker in terms of visited acceptance conditions than ac. 

When synchronizing with an edge of the property TGBA bearing ac leading to q, successive states of 
the Kripke will be aggregated as long as they model SF(q', ac). More formally, for a TGBA state q and a 
set of accepting condition ac C J", let us define 

SF{q,ac)= \/ f 

f,ac' 

q yq^S s.t. ac'<Zac 

Moreover, for o C r and / G B(AP), we define FSucc(a, f) = {s' & T \ 3s & a, s ^ s' e A h \{s) ^ /}. 
That is, first Filter a to only keep states satisiying /, then produce their Successors. The difference 
between SuccF and FSucc is whether the filter is applied on the source or destination states. Similarly 
to ReachF, we denote by FReach(a, /) the least subset of F satisfying both a C FReach(a, /) and 
FSucc(FReach(a, /), /) C FReach(a, /). 

Definition 8 (SLAP of a TGBA and a KS). Given a TGBA A = (AP', Q,J^,S,q°) and a Kripke 
structure T = {AP,r,X,A,so) over AP D AP' , the Self-Loop Aggregation Product of A and T is the 
TGBA denoted AM T {9, Q^,J^,5^,q^) where: 

- e^ = Qx(2^\{0}) 

3/ e ]B(AP') s.t. qi ^ q2ed, 
(/i = 92 =^ oc 7^ 0, and 
02 = FReach(FSucc(ai, /), SF(g'2, ac)) , 

9° =(gO,FReach({so},SF(gO,0))) 



(gi,ai) (92,02) 



Note that because of the way the product is built, it is not obvious what Boolean formula should label 
the edges of the SLAP product. Since in fact this label is irrelevant when checking language emptiness, 

we label all arcs of the SLAP with T and simply denote {qi,ai) (92,012) any transition {qi,ai) ^'"''^) 
(92,02) of the SLAP. 

Q X 2^ might seem very large but, as we will see in section 5, in practice the reachable states of the 
SLAP is a much smaller set than that of the product Qx F. Furthermore the FReach operation can be 
efficiently implemented as a symbolic least fix point. 

Fig. Ig represents the SLAP built from our example KS, and the TGBA of a U 6. The initial state 
of the SLAP iteratively aggregates successors of states verifying SF((j", 0) — ah. Then following the edge 

qO states are aggregated with condition SF((j'i,0) = _L. Hence 91 is synchronized with successors 

of states in {sq, Si, S2, S3, S4} satisfying b (i.e., successors of {54}). Finally, when synchronizing with edge 

9i "*"'*> 9i, we have SF(9i, {•}) = T, hence all states of the cycle {54, S5, sq, 57} are added. 



4.2 Proof of correctness 

As for the SOP, we aim at demonstrating that, given a KS and a TGBA, the emptiness of the language 
of the corresponding SLAP is equivalent to the emptiness of the language of the original synchronized 
product. This result is progressively demonstrated in the following by means of several intermediate 
lemmas. 

Lemma 6. Let A and T be defined as in Definition 8. Let (91,01) (92,02) & 5^ be a transition of 
the SLAP AMT ■ For any state S2 G 02 there exists at least one (possibly indirect) ancestor si G ai such 

that (91, Si) (92,^1) --^ {q2,t2) •••(92,^™) (92,^2) is a sequence of the synchronized product 
A<SiT with Mi, tt G 02, and Mi, a, C ac. 
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Fig. 3: A prefix ((ji,ai) ^> ((?2.o-2) (92, 02) of a run of some SLAP A^T (with different A 

and T from Fig. 1) is shown using big ehipses and bended arrows. The straight hues also shows the 
underlying connections between the states {qi,q2,q3. . . .} of the automaton A and between the states 
{si, S2, . ■ . , xi, X2 ■ ■ .} of the Kripke structure T that have been aggregated as ai, 02. a.-j • . • The acceptance 
conditions have been depicted as acj or and the labels of the transitions have been omitted for clarity. 
The dotted ellipses show the set of input states (7n(ai), In{a2), /n(a3)) as used in the proof of Lemma 6. 



For example consider transition (^1,01) {q2, 02) on Fig. 3, and some state in 02, say S2- Then si G Oi 
is an indirect ancestor of S2 s.t. (gi,si) (92,2:2) (92,52)- 

Proof. Let us define the set of input states of the aggregate 02 as In{a2) = {s' S 02 | 3s G ai, s — >■ s' S A\. 
This set cannot be empty since {qi,ai) (92, 012) • 

Consider a state S2 G 02. By construction of 02, 32 is reachable from some state in ti € In{a2), so 
there exists a path —>■ ^2 ^ S2 in the Kripke structure. 

By definition of if ti, t2 S2 belong to 02, the transitions between these states of T have been 

synchronized with self-loops 52 92 of A with ai C ac. Therefore the sequence ((72,^1) — ^ (92,^2) — ^ 
• • ■ (92, ^n) (92, S2) is a sequence of the synchronized product .4, (8) T. 

Moreover, since ti G /n(a2), there exists a state si in ai such that {qi,si) (92,^1)- 

Consequently, the path {qi, si) (92, ii) — ^ (92, ^2) — ^ • • • (92, in) (92, S2) satisfies the lemma. 

□ 

Lemma 7. If there exists a G Acc(^ ^T) an infinite run accepted by the SLAP, then there exists an 
accepting run it G Acc(^ (g) T) in the classical product. 

Proof. Let us denote cr = (gi, ai) (92, 02) — ^ (93, cts) — ^ • • ■ an accepting run of A^T. Let us 

build an infinite tree in which all nodes (except the root) arc states oi A(E)T. Let us call T the root, at 
depth 0. The set of nodes at depth n > is exactly the finite set of pairs {{qn, s) \ s £ a„} C Q x 7^. 

The parent of any node at level 1 is T. For any i > 0, the parent of a node (9j+i, s') with s' G Oj+i is 
the node {qi, s) for is any state s G such that {qi, s) is a (possibly indirect) ancestor of (9i+i, s') such 
that we observe aCj on the path between these two states. We know such a state {qi, s) exists because of 
Lemma 6. As a consequence of this parenting relation, every edge in this tree, except those leaving the 
root, correspond to a path between two states of A^T- 

Because the set of nodes at depth n > is finite, this infinite tree has finite branching. By Konig's 
Lemma it therefore contains an infinite branch. By following this branch and ignoring the first edge, we 
can construct a path of A ® T that starts in (171, si) for some ,si G ai, and that visits at least all the 
acceptance conditions aCj of a in the same order (and maybe more). To prove that this accepting path 
we have constructed actually occurs in a run of A <SiT, it remains to show that (91, si) is a state that is 
accessible from the initial state of ^ ® T. 

Obviously qi = 9° because (gi, ai) = q^ is the initial state of A^T. Furthermore we have si G ai, 
so by definition of q^, (9", si) must be reachable from (or equal to) (9°, sq) A<SiT. □ 

Lemma 8. For a given n and a finite path 7r„ — {qo, Sq) '^°'°°°> (91, si) • • • ^' — ((/„, s„) of A®T , 

there exists a finite path (j„ = {q'Q,ao) — (9i, «!)••• — {q'^,am) of A^T, with m < n, 
In = Qm' G ttm aud (/?„ : {0, . . . , m — 1} — >■ {0, . . . , TI — 1} IS a strictly increasing function such that 
Vj {3i, ipn{i) = j <s=^ aCi ^ 0). 

Proof. Let us prove this lemma by induction on n. It is true if n = 0: Given ttq = (gojSo), the path 
fo = (9'0,ao) = 9^1 = (90, FReachdso}, {A(so)} n X{qo,9)) satisfies the conditions (with being a null 
function). 
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Let us now assume that the lemma is true for n + 1 assuming it is true for n. Given a path 7r„+i = 

7r„ '^"'°^"> {qn+i, we know by hypothesis that we have a matching (t„ for 7r„. Let us consider how 

to extend (7„ into c7„+i to handle the new transition (g^, s„) '^"'°'^'') {Qn+i, Sn+i) of 7r„+i. 
There are two cases to consider: 

1. If Qn = q-n+i and acc„ = and A(.s„_|_i) |= SF((7„, ac), then by definition of FSucc and SF the last 
state of a„, {q'miQ"m) is such that Sn+i G am and q'm — In = Q'n+i- In that case On+i = cr„, and 

fn+l = fn- 

2. If qn ^ qn+i or ucCji 7^ or A(5^_|-i) ^ SF(^y^, etc), then because A(syi) ^ and ^ Sy^-i-i, by 

definition of (5^ there exists (g^, a™) ^ am+i) such that s„+i G Um+i and g^^^i = (?n+i. In 

this case, we can define (7„+i = i7„ °'^'^"> {q'm,_^_i,am+i) with Vz < n, (pn+i{i) = ^Pn{i) and <^„+i(n) = n. 

So by induction this lemma is true for all n e IN. □ 

Lemma 9. If there exists an infinite path tt € Acc(.4, ® T) accepting in A®T. Then there exists an 
accepting path in AMT as well. 

Proof. A^T has a finite number of states, so if Acc(^ ®T) ^ % then it contains at least one infinite 
path TT € kcc{A^T) that can be represented as a finite prefix followed by a finite cycle that is repeated 
infinitely often. 

Lemma 8 tells us that any prefix 7r„ of tt corresponds to some prefix fT„ of a path in ^ KIT in which the 
acceptance conditions of 7r„ occur in the same order. We have |(T„| < |7r„| ~ n but because tt will visit all 
acceptance conditions infinitely often, and these transitions will all appear in cr„ (only transition without 
acceptance conditions can be omitted from 5^), we can find some value of n for which |(t„| is arbitrary 
large. Because |(t„| can be made larger than the size of the SLAP, at some point this finite sequence will 
have to loop in a way that visits the acceptance conditions exactly in the same order as they appear in 
the cycle part of tt. By repeating this cycle part of C7„ we can therefore construct an infinite path a that 
is accepted by ^ lEl T. □ 

Theorem 3. Let A he a TGBA, and T he a Kripke structure. We have 

Acc(^ r) 7^ ^ Acc(^ H T) 7^ 

In other words, the SLAP of A and T accepts a run if and only if the synchronized product of these two 
structures accepts a run. 

Proof. follows from Lemma 7; follows from Lemma 9. □ 



4.3 Mixing SLAP and Fully Symbolic Approaches 

This section informally presents a variation on the SLAP algorithm, to use a fully symbolic algorithm in 
cases where the automaton state will no longer evolve. 

The principle is the following: when the product has reached a state where the formula automaton 
state is terminal (i.e., it has itself as only successor), we proceed to use a fully symbolic search for an 
accepted path in the states of the current aggregate. This variant is called SLAP-FST, standing for Fully 
Symbolic search in Terminal states. Note that we suppose here that such a terminal state allows accepting 
runs, otherwise semantic simplifications would have removed the state from the TGBA. 

In this variant, if qi is a terminal state, i.e., $q\ '^'°'^> q2 G 5, with qi ^ 52, a state (^1,01) of the 
product has itself as sole successor through an arc labeled (T, J^) if and only if ai admits a solution 
computed using a fully symbolic algorithm, or has no successors otherwise. 

The fully symbolic search uses the self-loop arcs on the formula TGBA state to compute the appro- 
priate transition rclation(s), and takes into account possibly multiple acceptance conditions. 

The rationale is that discovering this behavior when the aggregate is large, and particularly if there 
are long prefixes before reaching the SCC that bears all acceptance conditions, tends to create large SLAP 
structures in explicit size. The counterpart is that when no such solution exists, the fully symbolic SCC 
hull search may be quite costly. 

In practice this variation on the SLAP was proposed after manually examining cases where SLAP 
performance was disappointing, typically because the SLAP was much larger in explicit size than the 
SOP. As discussed in the performance section, this variation is on average more effective than the basic 
SLAP algorithm. 
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5 Experimentations 

In this section we present experimental results comparing several algorithms for hybrid or fully symbolic 
LTL model-checking. We use two different benchmark sets, one based on Petri net models using randomly 
generated LTL formulas, and one based on BEEM (Benchmarks for Explicit Model checkers [18]) models 
using meaningful LTL properties. We first present the context of these experiments, then focus on the 
results for Petri nets before detailing results for BEEM. 

5.1 Implementation 

We have implemented these new techniques (SOP, SLAP and SLAP-FST), the SOG [11,16] as weU as 
the classical fully symbolic algorithms (OWCTY [15] and EL [8]) and the hybrid approach of Biere et 
al. [2] (noted BCZ in the following) to allow comparisons. The software, available from ddd.lip6.fr, 
builds upon three existing components: Spot, SDD/ITS, and LTSmin. 

Spot (http://spot.lip6.fr) is a model checking library [7]: it provides bricks to build your own 
model checker based on the automata theoretic approach using TGBAs. It has been evaluated as "one of 
the best explicit LTL model-checkers" [20] . Spot provides translation algorithms from LTL to TGB A, an 
implementation of a product between a Kripke structure and a TGB A (def. 3), and various emptiness- 
check algorithms to decide if the language of a TGB A is empty (among other things). The library uses 
abstract interfaces, so any object that can be wrapped to conform to the Kripke or TGBA interfaces can 
interoperate with the algorithms supplied by Spot. 

SDD/ITS (http://ddd.lip6.fr) is a library representing Instantiable Transition Systems efficiently 
using Hierarchical Set Decision Diagrams [23]. ITS are essentially an abstract interface for (a variant of) 
labeled transition systems, and several input formalisms are supported (discrete time Petri nets, automata, 
and compositions thereof). SDD are a particular type of decision diagram that a) allow hierarchy in 
the state encoding, yielding smaller representations, b) support rewriting rules that allow the library 
to automatically [12] apply the symbolic saturation algorithm [4]. These features allow the SDD/ITS 
package to offer very competitive performance. 

LTSmin^ [3] is a tool allows to build a symbolic representation of the transition relation of a system 
using an explicit firing engine in background. The tool supports a wide range of input formalisms and 
explicit engines. For our experiments, we used LTSmin to build ETF files representing the transition 
relation. These files are then our input model, they were wrapped to conform to the ITS interface, thus 
allowing to apply our algorithms to any of the formalisms accepted by LTSmin or by ITS. We used the 
DVE variant of LTSmin to process the models from the BEEM benchmark. As noted by Blom et al. [3] 
this benchmark is not particularly favorable to symbolic approaches. 

The fully symbolic OWCTY algorithm is implemented directly on top of the ITS interface; it uses 
an ITS representing the TGBA derived from the LTL formula by Spot composed (at the ITS formalism 
level) with the ITS representing the system. The resulting ITS is then analyzed using OWCTY with the 
forward transition relation. 

The SOG is implemented as an object conforming to Spot's Kripke interface. It loads an ITS model, 
then builds the SOG on the fly, as required by the emptiness check of the product with the formula 
automaton. 

Both SOP and SLAP are implemented as objects conforming to Spot's product interface. The SOP 
and the SLAP classes both take an ITS model and a TGBA (the formula automaton) as input parameters, 
and build their specialized product on the fly, driven by the emptiness-check algorithm. 

5.2 Benchmark description 

We use here classic scalable Petri net examples taken from Ciardo's benchmark set [4]: slotted ring, 
Kanban, flexible manufacturing system, and dining philosophers. Table 1 gives the size of each model. 

The formulas considered include a selection of random LTL formulas, which were flltered to have a 
(basic TGBA/Kripke) product size of at least 1000 states. We also chose to have as many verified formulas 
(empty products) as violated formulas (non-empty products) to avoid favoring on-the-fly algorithms too 
much. To produce TGBA with several acceptance conditions, this benchmark includes 200 formulas for 
each model built from fairness assumptions of the form: (G Fpi A G Fp2 • • •) =^ ^■ 

We also used 100 random formulas that use the next operator, and hence are not stuttering invariant 
(these where not used for SOG that does not support them). 



® http : // f mt . cs . utwente . nl/tools/ltsmin 
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model 


state space 


model 


state space 


fms5 


2.9 X 10" 


kanban5 


2.5 X 10" 


fmslO 


2.5 X 10^ 


kanbanlO 


1 X 10" 


philolO 


4.6 X 10** 


ringG 


5.8 X 10^ 


philoSO 


2.3 X 10^^ 


ring? 


6.2 X 10® 


philolOO 


5.1 X lO"^"^ 


ringlO 


8.3 X 10" 



Table 1: Number of reachable states in the selected models. 



OWCTY EL BCZ SOG SOP SLAP SLAP-FST 

empty Win 103 (3%) 173 (5%) 53 (1%) 161 (4%) 735 (22%) 1256 (38%) 1703 (52%) 

X (3229 cases) Lose 260 (8%) 272 (8%) 2909 (90%) 481 (14%) 256 (7%) 246 (7%) 94 (2%) 

I Fail 221 (G'/() 25:', (7%.) 178() (55'/( ) 302 (9'/( ) 2i9(G'/() 2i:i ((>%.) 87(2%.) 

non empty Win 2 (0%) 10 (0%) 196 (4%) 513 (12%) 645 (15%) 2393 (59%) 1293 (31%) 

'5 (4048 cases) Lose 1846 (45%) 1378 (34%) 1924 (47%) 305 (7%) 318 (7%) 70 (1%) 40 (0%) 

Fail 804 (19%) 818 (20%) 1070 (26%) 263 (6%) 275 (6%) 69 (1%) 33 (0%) 

empty Win 14 (1%) 13 (1%) 2 (0%) 810 (73%) 823 (74%) 

^ (1108 cases) Lose 21 (1%) 21 (1%) 1081 (97%) 1 (0%) 1 (0%) 



Fail (0%) (0%) 355 (32%) (0%) (0%) 



•| non empty Win 12 (0%) 7 (0%) 778 (23%) 1912 (57%) 1872 (55%) 

(3348 cases) Lose 1697 (50%) 1627 (48%) 470 (14%) 54 (1%) 48 (1%) 

Fail 257 (7%) 272 (8%) 129 (3%) 29 (0%) 29 (0%) 

Table 2: On all experiments (grouped with respect to the existence of a counterexample and the use of 
a X operator in the LTL formula), we count the number of cases a specific method has (Win) the best 
time or (Lose) it has either run out of time or it has the worst time amongst successful methods. The 
Fail line shows how much of the Lost cases were timeouts. The sum of a line may exceed 100% if several 
methods are equally placed. 



We killed any process that exceeded 120 seconds of runtime, and set the garbage collection threshold 
at 1.3GB. Cases where all considered methods performed under 0.1s were filtered out from the results 
presented here: theses trivial cases represent only 4.2% of the entire benchmark, and were too fast too 
be allow any pertinent comparison. 



5.3 General comparison 

Table 2 gives a synthetic overview of the results presented hereafter. SLAP or SLAP-FST are the fastest 
methods in over half of all cases, and they are rarely the slowest. Furthermore, they have the least failure 
rate. This tabic also shows that BCZ has the highest failure rate and that the fully symbolic algorithms 
(OWCTY, EL) have trouble with non-empty products. 

Table 2 presents only the best and the worst methods. While Fig. 4 and 5 allow to compare the 
different methods in a finer manner. 

For each experiment (model/formula pair) we first collect the maximum time reached by a technique 
that did not fail, then compute for the other approaches what percentage of this maximum was used. 
The vertical segments visible at 100% thus show the number of runs for which this technique was the 
worst of those that did not fail. Any failures are plotted arbitrarily at 120%. This gives us a set of values 
between 0% and 120% for which we plot the cumulative distribution function. For instance, if a curve 
goes through the (20%, 2000) point, it means that for this technique, 2000 experiments took at most 20% 
of the time taken by the worst technique for the same experiments. 

The behavior at 120% represents the "Fail" line of previous table, while the behavior at 100% repre- 
sents the difference between the "Slow" and "Fail" lines ("Slow" methods include methods that failed). 

The left plot of Fig 4 for the non-empty cases shows that the on-the-fly mechanism allows all hy- 
brid algorithms (SLAP, SLAP-FST, SOG, SOP, BCZ) to outperform the symbolic ones (OWCTY, EL). 
However as seen previously, BCZ still fails more often than other methods. The SLAP and SLAP-FST 
method take less than 10% of the time of the slowest method in 80% of the cases. On left of Fig 5, the 
same effect is visible, although BCZ actually has less failures than the fully symbolic algorithms. 

The right plots for the empty cases show that fully symbolic algorithm behave relatively far better 
(all methods have to explore the full product anyway). BCZ spends too much time exploring enormous 
products, and timeouts. 
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Fig. 5: Cumulative plots comparing the time of all methods (except SOG and SOP) for non stuttering 
invariant properties. Non-empty products are shown on the left, and empty products on the right. 



For stuttering-invariant properties SLAP-FST and SLAP have similar performance, with a slight edge 
for SLAP-FST when the product is empty; however on Fig 5 SLAP and SLAP-FST are not significantly 
different. 

EL appears slightly superior to OWCTY in the non-empty case, while they have similar performances 
in the empty case. This is mostly discernible on stuttering-invariant properties. 

SOG and SOP show good results when there is a counterexample, and they perform better than BCZ 
in most cases. However SOG and SOP only support stuttering-invariant properties. As shown in Fig 5 
BCZ is a good alternative to fully symbolic algorithms in presence of a counter-example; it is however 
systematically outperformed by the new algorithms we propose in this paper. 



5.4 SLAP versus SLAP-FST 

To study the differences between SLAP and SLAP-FST consider the scatter plots from Fig. 6. The 
performances are presented using a logarithmic scale. Each point represents an experiment, i.e., a model 
and formula pair. We plot experiments that failed (due to timeout) as if they had taken 360 seconds, so 
they are clearly separated from experiments that didn't fail (by the wide white band). 

In these plots we have 11733 experiments, of which 132 proved too hard to solve for either algorithm 
within the time limit. OveraU SLAP algorithm solved 17 problems that SLAP-FST did not, and SLAP- 
FST solved 179 instances that SLAP did not. SLAP was at least a hundred times slower than SLAP-FST 
in 5 cases, ten times slower in 50 cases, and twice as slow in 164 cases. SLAP-FST was one hundred times 
slower in 3 cases, ten times slower in 34 cases, and twice as slow in 396 cases. 
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Fig. 6: Comparison of SLAP-FST against SLAP. Top left: time (in seconds); top right: memory (in 
kilobytes); bottom: product size (in states). 
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SLAP is on the average faster and consume less memory than SLAP-FST for non-empty products, 
but fails more often. SLAP-FST is better overall for empty products. Indeed the explicit product size of 
SLAP-FST is always smaller than that of SLAP, and often by several orders of magnitude. In some cases 
the SLAP degenerates to a state-space proportional to size of the explicit product while the SLAP-FST 
is able to keep the symbolic advantage. 

The cumulative plots of Fig. 7 make this advantage even more visible. Indeed at the cost of slight 
memory overhead, and a more significant overall time overhead (when counter-examples are present), 
SLAP-FST produces much smaller explicit structures (hence wins significantly for empty products). 

5.5 SLAP-FST versus other techniques 

In Fig. 8 and 9 we compare SLAP-FST against the other methods, using the same kind of logarithmic 
scatter plots. These plots only use stuttering invariant properties so they can be more easily compared. 
Unsurprisingly, the only methods that appear competitive are SOG and SOP; but to the advantage of 
SLAP-FST, SOG and SOP are not able to handle non stuttering-invariant properties. 

5.6 Fully symbolic algorithms: EL vs. OWCTY 

These two algorithms are worth comparing because they differ only is in the way that the acceptance 
conditions are alternated throughout the fixpoint computation. 

In Fig. 10, we have 11738 experiments, of which 1225 proved too hard to solve for either algorithm 
within the time limit. Overall EL algorithm solved 58 problems that OWCTY did not, and OWCTY 
solved 119 instances that EL did not. EL was at least ten times slower than OWCTY in 4 cases, and at 
least twice as slow in 141 cases, whereas OWCTY was twice as slow in 91 cases. 

Overall these plots show very little perceptible difference for non-empty products and seem to slightly 
favor OWCTY for empty products. Given the overall aspect of these plots that do not stray much from the 
diagonal, we can state that both algorithms have comparable empirical complexities on this benchmark. 
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Fig. 8: Comparison of SLAP-FST against BCZ, EL, and OWCTY in time (left) and memory (right). 
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Fig. 10: Performances in time (left, in seconds) and memory (right, in kilobytes) of fully symbolic algo- 
rithms for 11738 experiments. 



Although the scatter plot does not highhght this fact very blatantly, the density of experiments where 
EL outperformed OWCTY is actually quite high. The cumulative plots from Fig. 11 make this more 
visible. They also show that the difference between the two algorithms only very rarely exceed a ratio of 
2. 




5.7 Hybrid stuttering invariant algorithms: SOG vs. SOP 

We compare here the SOP and SOG algorithms. 

In these plots (Fig. 12), we have 7277 experiments, of which 399 proved too hard to solve for either 
algorithm within the time limit. Overall SOG algorithm solved 95 problems that SOP did not, and SOP 
solved 166 instances that SOG did not. SOG was at least a hundred times slower than SOP in 11 cases, 
ten times slower in 132 cases, and twice as slow in 1326 cases. A contrario, SOP was ten times slower 
than SOG in 13 cases, and twice as slow in 280 cases. 

Overall these plots show that SOP significantly outperforms SOG in many problem instances, partic- 
ularly when the full product needs to be built (i.e., it is empty so on-the-fly mechanism does not come 
into play). 
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Fig. 12: Performances in time (top-left, in seconds) and memory (top-right, in kilobytes) of observation 
graph algorithms for 7277 experiments, and product size (bottom). The product size show the number 
of states of the SOP against the number of state of the product between the Kripke structure and the 
SOG. 



The following cumulative plots (Fig. 13) make this more visible. It also shows that SOP does not 
necessarily outperform SOG, particularly when the product is non-empty. In fact the SOP may in some 
cases be much larger in explicit size than the SOG. This is shown in the bottom graph of figure 12. 

5.8 BEEM models 

We performed extensive experimentations using the BEEM (Benchmarks for Explicit Model checkers [18]) 
models and LTL formulas. 

The BEEM database contains a large set of examples modeling various network protocols, mutual 
exclusion or consensus problems. We used all the examples for which LTL formulas are provided, and ran 
the verification for both the formula and its negation, to increase the number of formulas. 

Surprisingly enough, all LTL formulas provided by BEEM are stuttering invariant. Thus we also 
generated a few random formulas which are not stuttering invariant. This benchmark is interesting as it 
shows some concurrent software oriented examples, and real formulas. However, the number of formulas 
is quite limited with respect to the previous benchmark. These formulas are also simpler, hence less able 
to discriminate the various algorithms that depend on the formula automaton. The number of reachable 
states in these models is also much lower than in the Petri net examples; this makes it more difficult to 
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Fig. 13: Compared cumulative performances in time of hybrid stuttering-invariant algorithms with (4048 
experiments, left) or without counterexamples (3229 experiments, right). 



measure the impact of large Kripke structures on the explicit size of hybrid algorithms. The transition 
relation, which is built by interacting with the LTSmin package is also less efficient than that of the Petri 
net benchmark, and takes less advantage of the automatic saturation features [12] of the SDD library 
that are heavily sollicited in the SOG, SOP and SLAP algorithms. 

A total of 729 formula/model pairs were computed for each algorithm, of which 292 are not stutterin 
invariant and not used for SOG or SOP. We filtered out model/formula pairs that took less than 0.1 
seconds to solve for all methods. 

Table 3 gives a synthetic overview of the results presented hereafter and Fig. 14 details these measures 
with a cumulative distribution function plot. 

For stuttering-invariant examples, when the product is empty SLAP or SLAP-FST are the fastest 
methods in over half of all cases, and they are rarely the slowest. Furthermore, they have the least failure 
rate, whether the product is empty or not. This table also shows that BCZ has the highest failure rate 
when the product is empty, although its is the fastest method in one third of cases whether the product 
is empty or not. SOG and SOP perform honorably when the product is non-empty (hence the on the 
fly mechanism comes into play), but behave quite porrly in the empty product case. The fully symbolic 
algorithms (OWCTY, EL) have trouble with non-empty products, but have a low failure rate when the 
product is empty though they rarely win. On this benchmark set (like on the Petri net measures) EL 
seems to perform slightly better than OWCTY. 

For non stuttering-invariant properties (which were randomly generated), BCZ behaves very well 
whether the product is empty or not. In non-empty case, as in our other measures, fully symbolic algo- 
rithms EL and OWCTY behave poorly with a lot of failures and slowest runtimes. SLAP and SLAP-FST 
are outperformed by BCZ on this benchmark set, but remain competitive. As shown in the cumulative 
distribution plot of Fig. 14(bottom), SLAP and SLAP-FST overtake the BCZ curve around SOsignificant 
number of problem instances has very good performance (as shown by the steep start for BCZ curve), 
but then flattens out, while the slope of SLAP and SLAP-FST is more regular. SLAP and SLAP-FST 
seem to perform more poorly on this benchmark set than in our other measures, however the number of 
experiments is much more limited here. 

The performances are presented as scatter plots using logarithmic scale. Each point represents an 
experiment, i.e., a model and formula pair. We killed any process that exceeded 800 seconds of runtime; 
hence for some formulas we were not able to compute the answer. We plot experiments that failed (due 
to timeout) as if they had taken 2400 seconds, so they are clearly separated from experiments that didn't 
fail (by the wide white band). 

The comparison between SOP and SOG slightly favors SOP for these examples. The two methods 
often have very similar complexity, as shown by the numerous points on the diagonal. These correspond to 
cases where the alphabet was not significantly reduced during the verification. Overall SOP outperforms 
SOG in practically all problem instances. Since SOP is similar but overall better than SOG, we compare 
other methods to SOP rather than SOG in the other plots. 

The comparison between SOP and EL favors SOP for non empty products (unsatisfied formulas) and 
EL for empty products (satisfied formulas). This means that the on-the-fly mechanism of SOP often allows 
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OWCTY EL BCZ SOG SOP SLAP SLAP-FST 

empty Win 2 (2%) 3 (3%) 31 (31%) 3 (3%) 1 (1%) 30 (30%) 40 (40%) 
X (100 cases) Lose 22 (22%) 10 (10%) 61 (61%) 26 (26%) 25 (25%) 3 (3%) 2 (2%) 

I Fail 8 (8%) 7 (7%) 38 (38%) 16 (16%) 16 (16%) 2 (2%) (0%) 

■S non empty Win (0%) (0%) 106 (32%) 24 (7%) 101 (30%) 70 (21%) 57 (17%) 
(329 cases) Lose 152 (46%) 199 (60%) 48 (14%) 28 (8%) 21 (6%) 19 (5%) 23 (6%) 

Fail 69 (20%) 72 (21%) 15 (4%) 16 (4%) 13 (3%) 19 (5%) 20 (6%) 

empty Win 10 (10%) 10 (10%) 47 (51%) 18 (19%) 30 (32%) 

(92 cases) Lose 37 (40%) 33 (35%) 21 (22%) 23 (25%) 14 (15%) 

^ Fail 4 (4%) 3 (3%) (0%) 4 (4%) 4 (4%) 

■| non empty Win 1 (0%) 2 (1%) 125 (62%) 39 (19%) 43 (21%) 

(200 cases) Lose 102 (51%) 108 (54%) 33 (16%) 15 (7%) 24 (12%) 

Fail 40 (20%) 40 (20%) 3 (1%) 14 (7%) 17 (8%) 

Table 3: On all experiments (grouped with respect to the existence of a counterexample and the use of 
a X operator in the LTL formula), we count the number of cases a specific method has (Win) the best 
time or (Lose) it has either run out of time or it has the worst time amongst successful methods. The 
Fail line shows how much of the Lost cases were timeouts. The sum of a line may exceed 100% if several 
methods are equally placed. 



BCZ 








EL 








OWCTY 








SLAP 








SLAP-FST 








SOG 








SOP 










































100 120 20 40 60 




Fig. 14: Cumulative plots comparing the time of all methods on the BEEM models. Non-empty products 
are shown on the left, and empty products on the right. Top for stuttering invariant properties and 
bottom for LTL formulae with the X operator. 
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Fig. 15: Performances in time for 721 experiments using the BEEM models. Plots with SOG or SOP only 
contain the 429 stuttering-invariant experiments. 
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to answer quite fast (when an accepting cycle exists), but when the full product needs to be explored EL 
is actually more effective than SOP. 

The comparison between EL and SLAP-FST clearly favors SLAP-FST in practically all experiments, 
with a significant portion of experiments that failed with EL but not with SLAP-FST. SLAP-FST like 
SOP benefits from the on-the-fly verification that often allows to answer without building the full product. 
But because the SLAP-FST is often quite small in explicit size, it performs quite well whether the product 
is empty or not. 

The plot comparing SOP to SLAP-FST is much more ambiguous. SLAP clearly outperforms SOP 
when the product is empty, but there are many problem instances where SOP find an accepting cycle 
faster. This could be due to the DFS order chosen during the on-the-fly verification, which makes these 
results diflScult to interpret, as either of the algorithms could be lucky. However, a significant number 
of problem instances were solved by SOP and not by SLAP-FST; hence overall both algorithms can be 
useful in practice. 

Comparison of SLAP-FST to BCZ, and of SOP to BCZ show that these three methods can be 
complementary. BCZ unfortunately has a large number of failures for empty products on stuttering 
invariant examples, hut can perform quite well in a significant number of problem instances. 

Comparison to the LTL model-checker provided by DiVine was attempted but when running in non 
compiled mode the run times are prohibitive, and when using the compiled mode the results are wrong 
on a significant number of problem instances, hence we have little confidence in the current distribution 
of DiVine. 

6 Conclusion and Perspectives 

We have presented two new hybrid techniques: the symbolic observation product (SOP), is a generalization 
of the symbolic observation graph (SOG) that diminishes the set of observed atomic propositions as we 
progress in the product, and the Self-Loop Aggregation Product (SLAP) that exploits the self-loops of the 
property automaton even if it does not express a stuttering formula. 

During our evaluation, we have found that SOP improves SOG, and outperform fully symbolic al- 
gorithms EL and OWCTY in the presence of counterexamples. SLAP surpasses both SOG (always), 
OWCTY (always), and SOP (when the product is empty). BCZ performs better than EL or OWCTy 
when the product is not empty and more poorly otherwise; the only set of experiments where it shows 
favorable results is the BEEM models with randomly generated formulas using the next LTL operator. 
When the product is not empty, SOP and SLAP techniques seem complementary. SLAP-FST provides 
the overall best performance and lowest failure rate of all the methods we compared. 

This work opens several perspectives. 

Firstly, the above two techniques replace the product used in the traditional automata-theoretic 
approach to model-checking in order to reduce the product graph while preserving the result of the 
emptiness-check. Another technique with the same goal is the Symbolic Synchronized Product (SSP) [1]. 
The SSP studies the symmetries of the model with respect to the current state of the property automaton, 
to aggregate symmetrically equivalent states. A classical emptiness-check of the SSP is possible, but Baarir 
and Duret-Lutz [1] also suggested two emptiness checks variants taking advantage of the inclusion between 
the aggregates. It would be interesting to see if a similar inclusion-aware emptiness checks could be used 
with SOG, SOP, and SLAP. 

Secondly, representing a stuttering property as a testing automata [13] is another way to take advan- 
tage of stuttering transitions in the model. In the product between a KS and a testing automaton, the 
latter does not move when the KS is stuttering. A possible perspective would be to adapt our stuttering- 
based techniques (SOG and SOP) to aggregate all states from the KS corresponding to one state of the 
testing automaton. 

Finally, since the SOG is a KS, and the SLAP is built upon a KS, it is possible to construct the SLAP 
of SOG. This is something we did not implement due to technical issues: in this case the aggregates are 
sets of sets of states. 
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